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ABSTRACT 

In the project described in this paper, the synthesis of 
research into situated learning was used to design and develop a multimedia 
learning environment to teach assessment strategies in mathematics to 
preservice teachers which was intended to transfer to teaching practice. The 
paper describes a follow-up study which investigated the influence of using 
the authentic approaches presented in the multimedia learning environment on 
the teaching and assessment practices of the same people as beginning 
teachers. Practicing teachers, who as preservice teachers participated in the 
earlier study, were interviewed with regard to the influence that the 
authentic approach had on their first and second years of teaching and 
assessing mathematics. Analysis of data indicates that a complex mix of 
influences and environmental factors impact upon the teachers* use of 
assessment strategies in their beginning years of teaching, and that 
preservice pedagogical beliefs are not always translated to beginning 
classroom practice. (Author) 
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In the project described in this paper, the synthesis of research into situated learning 
was used to design and develop a multimedia learning environment to teach assess- 
ment strategies in mathematics to preservice teachers which was intended to transfer 
to teaching practice. The paper describes a follow-up studywhich investigated the 
influence of using the authentic approaches presented in the multimedia learning envi- 
ronment on the teaching and assessment practices of the same people as beginning 
teachers. Practicing teachers, who as preservice teachers participated in the earlier 
study, were interviewed with regard to the influence that the authentic approach had 
on their first and second years of teaching and assessing mathematics. Analysis of data 
indicates that a complex mix of influences and environmental factors impact upon 
the teachers* use of assessment strategies in their beginning years of teaching, and 
that preservice pedagogical beliefs are not always translated to beginning classroom 
practice. 

Introduction 



^ The persistent failure of beginning teachers to access pedagogical skills and strat- 

Q egies acquired during teacher training is an ongoing problem for teacher educators 

txj throughout the world. In mathematics, despite the variety of strategies recommended, 

the Third International Mathematics and Science Study indicated that teachers gener- 
ally continue to limit their approaches to traditional methods (Lokan, Ford & Green- 
wood, 1996). Traditional classroom approaches reflect a view of mathematics as a set 
of discrete, hierarchically arranged facts and skills; a view of learning mathematics as 
replication and repetition; a view of teaching mathematics as exposition and practice; 
and a view of assessing mathematics as paper and pencil testing for the sole purpose 
of grading and ranking (Niss, 1993; Stephens, 1992). This lack of progress may result 
from ineffective teacher-training courses (Borko, et al., 1992). Even newly-trained 
teachers with non-traditional beliefs about mathematics teaching and learning return 
to traditional teaching approaches (Brown & Borko, 1992; Raymond, 1997). 

One reason for the long-term ineffective practical outcomes of such university 
courses, is according to Resnick (1987), a result of too little engagement with genuine 
situations, and too much emphasis on theoretical perspectives. Genuine situations such 
as school practice, may enable some student teachers to experience classrooms which 
reflect curriculum reform. For many, however, their school practicum experience is 
often a reaffirmation of the traditional approaches that are already ingrained from their 
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own schooling (Comiti & Ball, 1996). For those who do observe and engage with 
teachers using current approaches, the extent and range of approaches seen is limited 
to a few — the ones with which their supervising teachers are familiar and comfortable 
implementing. One way to align university teaching and learning more substantially 
with the way learning is achieved in real-life settings is to base instructional methods 
on more recent theories of learning which reflect this shift, such as situated learn- 
ing (e.g.. Brown, Collins, & Duguid, 1989; Collins, Brown, & Newman, 1989; Lave 
& Wenger, 1991; McLellan, 1996). Collins (1988) defines situated learning as: ‘the 
notion of learning knowledge and skills in contexts that reflect the way the knowledge 
will be useful in real life’ (p. 2). 

Nine critical characteristics of situated learning environments were defined from 
the extensive literature on the subject and used to inform the instructional design of 
learning environments (described in detail in Herrington & Oliver, 2000). The learning 
environment, in reflecting a situated learning approach, needed to: 

• provide an authentic context that reflects the way the knowledge will be used in 
real-life; 

• provide authentic activities; 

• provide access to expert performances and the modelling of processes; 

• provide multiple roles and perspectives; 

• support collaborative construction of knowledge; 

• provide coaching and scaffolding at critical times; 

• promote reflection to enable abstractions to be formed; 

• promote articulation to enable tacit knowledge to be made explicit; and 

• provide for integrated assessment of learning. 

A multimedia program was developed to enable teachers and pre-service teach- 
ers of mathematics to explore issues of assessment, entitled Investigating Assessment 
Strategies in Mathematics Classrooms (Herrington, Sparrow, Herrington, & Oliver, 
1997). Twenty three assessment strategies suitable for use in K-12 classrooms were 
identified and grouped in the following categories: Questioning, Interviewing, Test’ 
ing. Problem solving. Reporting and Self-assessment, and presented in a multimedia 
format. Five complex, authentic activities were designed consisting of memos and 
other documents, to enable students to explore the resource within the parameters and 
constraints of a realistic task. The interface (see Figure 1) simulates the front part of 
a classroom with the resources represented by appropriate metaphors (such as video 
cassettes, filing cabinet drawers, folders on the desk) giving direct access to the fol- 
lowing resources: 

1) Video clips of teachers using various assessment techniques 

2) Video clips of teachers’ comments on the strategies 
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3) Video clips of children’s comments on the strategies to present their own 
thoughts 

4) Interviews with experts in the field to provide theoretical perspectives 

5) Reflections by third year preservice teachers to provide practical advice 

6) Text descriptions of each assessment category 

7) Teachers’ resources and children’s work samples 

8) An electronic notebook to enable students to copy text and to write their own 
ideas 

9) Problems and investigations to enable the students to complete authentic tasks. 



Click here to read a 
description and 
discussion of the 
strategy 

Click here to see 
scanned images of 
samples of 
children’s work 
and other material 

Click here to read a 
pre service teacher's 
comments and advice 
on the strategy 

Click here to read an 
Interview with a 
mathematics expert 
on the strategy 




Choose a strategy to 
Investigate by clicking on one of 
the categories shown in black 
on the whiteboard 



Click on the up and down 
arrows to scroll through 
the list 




Click here to play 
a video cWp of 
the strategy 
’ being used in the 
classroom 

Click here to play 
a video clip of a 
teacher's 
comments on 
the strategy 



Click here to play 
a video clip of a 
student's 
comments on 
the strategy 



Click here to use the electronic notebook, 
which Includes space for reflections and 
pastes from the filing cabinet, it also 
includes iir/estlgatlons and problems 



Click here for balloon help 
_ on the functions of the 
objects on the interface 



Figure 1. The interface of Investigating Assessment Strategies in Mathematics Class- 
rooms showing constitutive elements. 



In the first study, preservice teachers explored a variety of assessment strategies 
appropriate to K-12 mathematics classrooms using the multimedia assessment pro- 
gram. The students spent three weeks of the semester examining strategies within the 
context of a complex and sustained authentic activity, where in small collaborative 
groups, they prepared and presented a report on a new assessment plan for mathemat- 
ics in a school. In order to do this, they were asked to consult with experts, look at what 
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was happening in classrooms, talk to teachers, talk to students and so on — all of which 
could be done ‘virtually’ from the CD-ROM. 

Three pairs of students were interviewed and observed as they used the multime- 
dia program and the analysis of talk revealed that students used a substantial amount of 
higher-order thinking as they worked with the assessment program (reported in Her- 
rington & Oliver, 1999). A study of the transfer of a variety of assessment strategies 
to classroom practice was conducted with the students as they completed two weeks 
professional practice in schools approximately five weeks after the completion of their 
work on the assessment program (reported in Herrington, Herrington, & Sparrow, 
2000). All the students were required to teach mathematics classes in this practice, and 
it was expected that they would have the opportunity to implement different assessment 
strategies. Both students and supervising teachers in the schools were interviewed and 
the comments were transcribed and analyzed. Transfer was thought to have occurred 
if firstly, students using the interactive multimedia program on assessment had a good 
understanding of the types of assessment appropriate in the mathematics classroom 
and were able to articulate this understanding; and secondly, they employed a variety 
of the assessment techniques shown in the program, as opposed to the predominant use 
of pencil-and-paper tests (Cognition and Technology Group at Vanderbilt, 1993). 

Analysis of the data shows that all the students could speak knowledgably and 
confidently about assessment, and all the students used a variety of techniques to 
assess children’s understanding. As was expected, all were influenced strongly by 
the supervising teacher in the schools, many of whom had planned assessment in 
advance of the students’ arrival. However, even when assessments had been planned 
in advance, all students used techniques that they were able to use without the contri- 
bution or agreement of the supervising teacher — such as checklists, anecdotal records 
and open interviews, which students could use individually with children as they cir- 
culated around classrooms. Five of the six students attributed their use of assessment 
techniques directly to the interactive multimedia program. 

These very positive findings were qualified by two mitigating factors: the brevity 
of a two week professional practice and the substantial influence of the supervising 
teacher. In the words of one student: “I’m not the qualified teacher. I’m in their situa- 
tion, in their room, conforming to their rules. So I can’t just suddenly say ‘Hey, let’s 
do some oral assessment’”. 

Many of the students in the first study were inhibited in the choice of assessment 
strategies by the influence and authority of their supervising teachers, in a way which 
may not have been an issue if the students were practising teachers with their own 
classes. In order to investigate this issue further, a follow-up study of those students 
who had gained employment was carried out when the students were in the second 
year of teaching. 

In the follow up study reported here, interviews were conducted with students 
from the first study who had gained employment as teachers. These four teachers were 
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in their second year of teaching, three in secondary schools Evie, Rowan and Zoe 
(pseudonyms used) and one in a primary school, Debra. Three teachers (two female, 
one male) were teaching mathematics in private schools in the metropolitan area of 
Perth, and one female, Evie, was teaching in a remote outback Government school in 
the north of Western Australia. 

Participants were interviewed at their schools, for approximately 90-120 minutes 
each using an interview protocol described by Denzin (1989) as a nori’Scheduled 
standardized interview. Questions and probes were determined in advance, but there 
was flexibility in adjusting the interview protocol. Students were questioned about: 
their beliefs about assessment; their knowledge and use of assessment strategies in 
mathematics; the influence of school policy, colleagues, and national guidelines on 
their assessment practices; and the influence of their teacher training on their current 
practices. The interviews were recorded and transcribed for analysis. Transcripts of 
interviews were analyzed using techniques of qualitative analysis recommended by 
Miles and Huberman (1994) and Eisner (1991). Transcripts were summarized for each 
student and placed in a matrix that allowed comparisons to be made for each student’s 
response to each interview question. A further summary of these responses is given 
below. 

Results 

Beliefs About Assessment 

The secondary teachers felt that generally assessment practices had changed 
since they were at school with more emphasis on problem solving, investigations, and 
project work. Evie believed that many students had difficulties in adapting to different 
types of assessment, particularly those students that lacked motivation and showed 
poor attendance. She noted that open book exams had become a common practice at 
her school. 

Zoe felt that the use of graphing calculators resulted in more conceptually based 
questions and was concerned about the lack of reliability of tests and grading. Rowan 
commented that there was too much testing in his school and would like to see more of 
an emphasis on investigations and problem solving with younger students. 

Debra, a year one primary school teacher, enjoyed the use of varied approaches 
to understanding students and felt that there was less use of worksheets than when she 
had gone to school. She believed her assessment to be diagnostic identifying areas 
where individual children needed assistance. 

Seif Reported Assessment Practices 

Two secondary teachers. Rowan and Zoe, reported that testing was used as their 
main form of assessment, ranging from multiple choice tests to conceptual tests using 
graphics calculators. They also indicated that they used group projects, posters and 
oral reports. The two other teachers, Evie and Debra, reported on using tests, observa- 
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tions, checklists, portfolios, projects and individual interviews. Debra and Rowan had 
used self assessment techniques where children reflected on their learning by drawing 
faces to express their understanding and by making review sheets for exams. Zoe indi- 
cated that she had tried using journals but found them unsuccessful because students 
found it difficult to give thoughtful reflections. 

Perceived Influences on Assessment Practices 

Debra relied heavily on the advice of a more experienced teacher and support 
from the Principal. Evie also felt that advice from colleagues was more helpful than 
professional development organized by the district office. 

Zoe attributed her assessment approaches to mathematics education courses com- 
pleted at university. However, the school had a policy of encouraging similar assess- 
ment strategies across grade levels which she indicated limited her use of alternative 
approaches. 

All the teachers recalled having used the interactive multimedia in their math- 
ematics education course and felt that it was a helpful and important resource - one 
with which they were able to learn about different forms of assessment in a collabora- 
tive and enjoyable way. Debra, in particular, felt that she would be able to find good 
use for the program now that she is teaching. 

Discussion 

In both studies the teachers appeared to have a sound understanding of alterna- 
tive assessment strategies and ways in which they could be implemented. While the 
first study presented constraints in employing these strategies on professional prac- 
tice, (such as limited time and the influence of the supervising teacher), as teachers 
they were also faced with cultural and practical constraints. Constraints include the 
requirements of the mathematics department in the school. State Curriculum guide- 
lines, issues of practical classroom management, personal issues such as the amount 
of time that can be devoted to planning, and convictions about the suitability of certain 
types of assessment for certain students. As such, the teachers’ pedagogical beliefs do 
not always translate to classroom practice. 

Analysis of data indicates that a complex mix of influences and environmental 
factors impact upon the teachers’ use of assessment strategies in their beginning 
teaching. For example, Zoe commenting on her practicum supervising teacher’s use 
of assessment strategies in 1996, appeared to have a range of alternative strategies in 
mind which she failed to use in her teaching practice in 2000: 

Interview with Zoe (1996): It’s opened my eyes a lot more . . . and also watch- 
ing my [supervising] teacher and really disagreeing with a lot of the assess- 
ment strategies he’d use. He only used pencil and paper assessment strate- 
gies. Of course I didn’t say anything, but I’d sit there thinking ‘Oh remember 
what we learnt’. 
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Interview with Zoe (2000): Testing. That’s the main approach. With years 8, 

9 and 10 mainly topic testing, and then at the end of their year we do an exam 
to get them ready for Year 1 1 and 12 ... If you put it on a piece of paper, you 
know if they can interpret it. I don’t think that sort of testing is fantastic for 
every student ... but I can’t think of another really appropriate assessment 
task. 

The enthusiasm Evie had in 1996 to try a range of assessment strategies in 
her own classes, by 2000 had been overwhelmed by significant social and 
cultural problems: 

Interview with Evie (1996): There were only limited types of assessment 
that I could use [on teaching practicum], but hopefully in the future I’ll be 
able to use a wider range of the ones that were on the multimedia. Hopefully 
I’ll be able to ... start journals and things like that. 

Interview with Evie (2000): I’d like to vary a lot of things, such as I’d like 
to do a lot more collaborative work with the kids, group work. And probably 
even presentation type stuff, where kids can actually demonstrate or explain 
their findings, whether it be an investigation or even project work ... I Just 
find that all these fantastic ideas that I come up with usually seem to back- 
fire when I use them in the classroom. I don’t see myself as being a really 
boring teacher . . . [and] I’m not blaming it on the kids — but they just lack any 
self-motivation. And it’s not Just me ... all the learning areas are having the 
same problems with the same kids. I think the problem is they don’t see the 
importance of education . . . there is a high rate of kids dropping out, and girls 
falling pregnant at a young age which is really sad. 

While the teachers’ pedagogical beliefs do not always translate to classroom prac- 
tice, this is not to conclude that teachers’ beliefs about assessment revert to traditional 
beliefs. All the teachers in the study, when prompted on various assessment strategies 
indicated they had tried alternative approaches and that they would be willing to try 
these techniques in the future. One teacher indicated that she would try Journals as an 
assessment strategy when she could better explain to the students how to keep them. 
The use of a situated learning environment (such as that provided on the CD-ROM) 
during teacher training may provide contextualised knowledge about assessment strat- 
egies but this knowledge will not necessarily be applied in teaching practice either 
as preservice or novice teachers because of intervening constraints. The scenarios 
aimed at developing pedagogical knowledge but did not directly focus on potential 
constraints that needed to be taken into account. This could be overcome by including 
these aspects as part of the problem that needed consideration, and could equally well 
be evident as teacher guidance when the student teachers discussed their solutions to 
the scenarios. 




PMEPTE 1111 



9/27/02, 11:31:57 AM 





m 



1112 Teacher Education 



The apparently powerful social, cultural and environmental pressures with which 
new teachers must deal — pressures which cause many to revert to more traditional 
modes of teaching to deal with them — could also arguably be ameliorated by provid- 
ing ongoing professional development. The professional development of beginning 
teachers is recognized as a necessary process in the long-term development of ‘full- 
fledged professionals’ (Schoenfeld, 2002, p. 22). Without support, these teachers can 
find it difficult to adapt to change, and can readily abandon approaches to teaching 
and learning emphasized in their university courses and current curriculum initiatives 
(Schoenfeld, 2002). 

However, such support needs to be focussed. It is apparent the teachers in this 
study retained their pedagogical beliefs and practices that they had learnt during their 
university mathematics education course using the multimedia but were often con- 
strained in applying this knowledge in the initial teaching environment. Professional 
development that simply revisits approaches encountered at university may well be 
ineffective because it does not address the range of constraints that are faced by novice 
teachers. On the other hand, novice teachers may benefit from support in constraint 
management where the problems of novice teachers could be aired and discussed in 
a similar way that could happen with preservice teachers. Rather than face to face 
discussion of potential problems, the Internet has the potential for providing a forum 
where constraints could be openly discussed and resolutions reached in a collegial 
environment. 

Conclusion 

The studies described in this paper have revealed an ongoing discrepancy between 
the preparation of teachers and the reality and demands of classroom teaching. The 
use of situated learning environments has gone a long way towards ameliorating the 
cognitive gap between theory and practice, but it has not been entirely successful (on 
its own) in promoting sustained transfer to the real teaching situation. Practical con- 
straints and real-life restrictions on teachers’ practice need to be accounted for more 
substantially and incorporated more fully into the situated learning model. Further 
research on these factors, and inservice professional development as described here, 
may help to ensure that pedagogically sound practice is learned, applied and sustained 
throughout teachers’ professional life. 
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